Calculators, as well as handheld computing devices, laptops, and desktop computers, typically display the results of a mathematical operation with as many digits as the device's display will allow. Hence, such devices assume that entered values for the mathematical operation are exact quantities having infinite precision. However, this assumption is incorrect if one or more of the values entered is based on a measurement, which has a finite precision.
For example, if the mathematical operation of 253.7÷7.64 is entered on a common calculator, the floating point answer displayed is 33.20680628. Even though the first operand (253.7) has only four significant digits or significant figures and the second operand (7.64) has only three significant figures, the floating point answer (33.20680628) is displayed with ten significant figures. If the first operand (253.7) and the second operand (7.64) are both measured quantities, then the proper number of significant figures for the answer to the mathematical operation is three—corresponding to the least number of significant figures for the operands in this example. In other words, the answer in this example should be rounded to 33.2 to reflect three significant figures. Thus, a person who does not understand the concept of significant figures may be misled to believe that the floating point answer reflects the actual level of precision of the answer.
As another example, if a first operand is a measured quantity of 23.625 and the second operand is a measured quantity of 3.0125 for a mathematical operation of 23.625+3.0125, a typical calculator will display a floating point result of 26.6375. Note that the floating point result has a precision to the {fraction (1/10,000)} place. However, the least precise operand of the mathematical operation is the first operand (23.625), which has a precision to the {fraction (1/1,000)} place. Hence, the properly rounded answer is 26.638—rounded to a precision of {fraction (1/1,000)}—corresponding to the least precise measured value. Thus, again, a person who does not understand the concept of rounding to the least precision for mathematical operations involving measured values may be mislead to believe that the floating point answer reflects the actual level of precision of the answer. Hence, there is a need for a tool that will automatically perform the proper rounding of an answer to a mathematical operation involving measured values, and that will inform or remind the user that the number has been rounded accordingly.
The concept of rounding a result of a mathematical operation to the proper precision or to the proper number of significant figures is a known concept. In applying the rules for rounding to the proper number of significant figures, the following rules are used to determine the correct number of significant figures for values in standard decimal notation (as opposed to scientific notation described further below):
ExampleNumber of SignificantRuleValueFigures for Example ValueNonzero digits are always112significant.5.7594Zeros between nonzero10.054digits are significant.900055Zeros in front of nonzero0.00031digits are not significant.0.05093Zeros at the end of a232number to the right of adecimal point are significant.23.00006Zeros at the end of a460002whole number are significantonly if the decimal point46000.5is shown.The rule used to determine the correct number of significant figures for values in scientific notation is that only significant figures are included when writing a number in scientific notation. For example, 3×106 has one significant figure, and 3.00×106 has three significant figures.
For a mathematical operation not having an addition or subtraction operation involved, the answer to the mathematical operation is rounded to the least number of significant figures corresponding to the measured value in the mathematical operation having the least number of significant figures. Such mathematical operations include multiplication and division. Squaring (e.g., 52=25) operations, which are essentially multiplication, and other operations raising a value to a power, are also included. For example in the mathematical operation of 12.257×1.36 (assuming all operands are measured values), the floating point answer is 16.66952. The first operand (12.257) has five significant figures and the second operand (1.36) has three significant figures. Hence, the operand with the least number of significant figures is the second operand having three significant figures. Thus, the answer rounded to the least number of significant figures (i.e., rounded to three significant figures) is 16.7.
For a mathematical operation having an addition and/or subtraction operation involved, the answer to the mathematical operation is rounded to the precision of the least precise value of the measured operands. For example in the mathematical operation of 3.95+213.6+2.879 (assuming all operands are measured numbers), the floating point answer is 220.429. The first operand (3.95) is precise to hundredths ({fraction (1/100)}), the second operand (213.6) is precise to tenths ({fraction (1/10)}), and the third operand (2.879) is precise to thousandths ({fraction (1/1000)}). Hence, the least precise operand of the mathematical operation is the second operand, which is precise to the nearest tenth. Thus, the floating point answer (220.429) should be rounded to the nearest tenth, which yields a rounded answer of 220.4. As another example, 29000+6.0 (assuming each operand is a measured value) yields a floating point result of 29006. Because the first operand (29000) is precise to thousands (1000) and the second operand is precise to tenth ({fraction (1/10)}), the properly rounded answer will be 29000 (rounded to the nearest thousands—the least precise of the mathematical operation).
For mixed mathematical operation on one or more measured values involving addition and/or subtraction as well as multiplication, division, and/or raising a number to a power, the final result is rounded according to the addition and subtraction rules (least precision) described above. In such a mixed mathematical operation, there is a default order to performing the mathematical operations, unless another order is specified. First, the multiplication, division, and/or raising a number to a power operations are performed within each group separated by an addition or subtraction operation. Second, the results of each group are added and/or subtracted accordingly. The floating point (unrounded) result for each group is maintained for the calculations and only the final result is rounded.
For example in the mixed mathematical operation of 12÷4.103+2.31×94.8 (assuming each operand is a measured value), the floating point answer is 221.9126893. The first operand (12) is precise to ones and has two significant figures, the second operand (4.103) is precise to the thousandths ({fraction (1/1000)}) and has four significant figures, the third operand (2.31) is precise to hundredths ({fraction (1/100)}) and has three significant figures, and the fourth operand (94.8) is precise to the tenths ({fraction (1/10)}) and has three significant figures. Thus the properly rounded answer (according to the least precise operand—the first operand) is 222, which is precise to the nearest ones and happens to have three significant figures. Therefore, the answer is rounded to the nearest ones according to the addition and subtraction rules, and the answer has three significant figures, even though the least number of significant figures among the operands was two significant figures.
When an exact value is involved in a mathematical operation, its number of significant figures does not affect the proper rounding of the answer. For example, in taking the average of three measured values 3.473, 23.937, and 102.54, the sum of these values is divided by three because there are exactly three values being averaged. Hence, the operand 3 in the mathematical operation does not affect the resulting precision of the properly rounded answer because it is an exact number. Hence, the resulting answer should be rounded to the nearest hundredths ({fraction (1/100)}) because the least precise measured value (102.54) is precise to hundredths. Therefore, the properly rounded answer is 43.32. Note that these methods of rounding the result of a mathematical operation based on the significant figures and/or precision of the operands are merely conventions to express results with the appropriate precision based upon the precision of the measured values, rather than firm rules. Hence, these methods may vary slightly.
Because these rules for rounding an answer for a mathematical operation involving one or more measured values are more easily understood through examples, there is a need for an education tool that will display the number of significant figures for each operand entered. Such a display can illustrate to the user, or reassure the user, of the number of significant figures for each operand, which may aid in the education process. Also, there is a need for an education tool that will determine and display a properly rounded answer to a mathematical operation involving one or more measured values. Such a tool can illustrate or reassure the user of the properly rounded answer, which may also aid in the educational process.